Using as a starting point the postulate that sensory systems have evolved to perform optimal transformations on behaviorally relevant or natural stimuli, we will use systems analysis methods and information theoretic principles to develop a theory of auditory processing. The goals of our theory will be to predict the stimulus-response transformations that are found at different stages of auditory processing. First, we will obtain theoretical predictions for the distribution of linear receptive fields by jointly maximizing signal to noise ratio and entropy in the output of the ensemble of filters when presented with natural sounds. Second, we will derive non-linear stimulus-response transformations that can be obtained with biologically plausible networks and that will minimize the mutual information across neurons. These neural networks will perform a form of independent component analysis, in which the resulting operation is to extract independent acoustical features in natural sounds. We will also develop novel methods to estimate the information transmitted by single neurons and ensembles of neurons in songbirds. The goodness of fit of the theoretical models will be assessed in two steps. First we will compare the theoretical stimulus-response functions with the functions of the same order that will be obtained directly from the neural data. Second, we will evaluate how well the stimulus-response functions describe the actual neural transformation by comparing predicted responses and predicted information rates with actual responses and measured information rates. Our analysis will give us insight on how complex natural sounds are processed in the auditory system of animals and humans. Understanding how biological systems process natural sounds will be instrumental in the development of novel algorithms in engineering applications for sound compression, speech recognition and sound pre-processing for hearing aids and auditory prosthetics.